In complex environment, existing pedestrian detection methods can not be very good to achieve high recall rate and efficient detection. To solve this problem, a pedestrian detection method based on Convolutional Neural Network (CNN) was proposed. Firstly, pedestrian locations in input images were initially detected with single step detection upgrade network (YOLOv2) derived from CNN. Secondly, a network with target classification and bounding box regression was designed to cascade with YOLOv2 network, which made reclassification and regression of pedestrian location initially detected by YOLOv2, to reduce error detections and increase recall rate. Finally, a Non-Maximum Suppression (NMS) method was used to remove redundant bounding boxes. The experimental results show that, in INRIA and Caltech dataset, the proposed method increases recall rate by 3.3 percentage points, and the accuracy is increased by 5.1 percentage points compared with original YOLOv2. It also reached a speed of 11.6FPS (Frames Per Second) to realize real-time detection. Compared with the existing six popular pedestrian detection methods, the proposed method has better overall performance.